Random Feature Maps for Dot Product Kernels Supplementary Material

This document contains detailed proofs of theorems stated in the main article entitled Random Feature Maps for Dot Product Kernels. 1 Proof of Theorem 1 We first recollect Schoenberg’s result in its original form Theorem 1 (Schoenberg (1942), Theorem 2). A function f : [−1, 1]→ R constitutes a positive definite kernel K : S∞ × S∞ → R, K : (x,y) 7→ f(〈x,y〉) iff f is an analytic function admitting a Maclaurin expansion with only non-negative coefficients i.e. f(x) = ∞ ∑ n=0 anx , an ≥ 0, n = 0, 1, 2, . . .. Here S∞ = {x ∈ H : ‖x‖2 = 1} for some Hilbert space H. Corollary 2 (Theorem 1 restated). A function f : R → R constitutes a positive definite kernel K : B2 (0, 1) × B2 (0, 1) → R, K : (x,y) 7→ f(〈x,y〉) iff f is an analytic function admitting a Maclaurin expansion with only non-negative coefficients i.e. f(x) = ∞ ∑ n=0 anx , an ≥ 0, n = 0, 1, 2, . . .. Here B2 (0, 1) ⊂ H for some Hilbert space H. Proof. To see that the non-negativeness of the coefficients of the Maclaurin expansion is necessary just apply Theorem 1 to points on S∞. Since {〈x,y〉 : x,y ∈ B2 (0, 1)} = {〈x,y〉 : x,y ∈ S∞}, the result extends to the general case when the points are coming from B2 (0, 1). To see that this suffices we make use of some well known facts regarding positive definite kernels (for example refer to Schölkopf and Smola, 2002). Fact 3. If Kn, n ∈ N are positive definite kernels defined on some common domain then the following statements are true 1. cmKm + cnKn is also a positive definite kernel provided cm, cn ≥ 0. 2. KmKn is also a positive definite kernel. 3. If lim n→∞ Kn = K and K is continuous then K is also a positive definite kernel. Starting with the fact that the dot product kernel is positive definite on any Hilbert space H, applying Fact 3.1 and Fact 3.2, we get that for every n ∈ N , the kernel Kn(x,y) = n ∑ i=0 ai 〈x,y〉 is positive definite. An application of Fact 3.3 along with the fact that the Maclaurin series converges uniformly within its radius of convergence then proves the result.